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ABSTRACT 



This paper provides an overview of telephone survey research 
by colleges and universities and reports on comparisons between mail survey 
respondents and non- respondents who were later contacted by telephone. After 
an introduction, the paper discusses the various steps of a telephone survey 
project. These include: (1) identifying the population to be surveyed; (2) 

selecting the sampling methodology (usually either the standard random 
sampling or the random number dialing method) ; and (3) designing the 
telephone survey (focusing on the essential information and how it will be 
used) . The paper emphasizes the importance of careful planning, designing the 
survey format, training of interviewers, and spot checking of interviewing 
techniques. It also summarizes results of comparisons between mail survey 
respondents and nonrespondents who were later telephoned for an interview. 
These comparisons were conducted on results of annual studies of the 
college's career program completers. Although nonrespondents to the mail 
surveys later contacted by telephone tended to be younger and more often male 
than female, there were no notable differences in the level of career success 
and satisfaction reported by mail survey respondents and nonrespondents who 
completed the telephone version of the survey. ( Author /BF) 
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Abstract 

The rise to prominence of surveys by telephone has been attributed to advances in 
telephone technology, improvement of telephone research procedures, near complete 
accessibility of any population via the telephone, expansion of a survey industry that requires 
quality data generated in an efficient and timely manner for eager clients, and availability of 
considerable information, based on experiment and experience, on what techniques work or do 
not work when doing telephone surveys. This paper details 10 years of experience with 
telephone survey methodology utilized to collect data from business and industry, former 
students, and individuals residing in the college’s service area. 

Of particular interest to researchers are the results of comparisons between mail survey 
respondents and nonrespondents who were later telephoned for an interview. These 
comparisons were conducted on results of annual studies of the college’s career program 
completers. Although nonrespondents tended to be younger and more often male than female, 
there were no notable differences in the level of success and satisfaction reported. 




3 



4 



Telephone Surveys 



I 



Telephone Survey Research: An Overview 

Introduction 

In the history of survey research, no other technical or procedural innovation, with the 
exception of the computer, has made as significant an impact on this type of data gathering as 
the telephone. The telephone survey is one of the dominant and most popular survey 
techniques in use today. Most commercial and some academic survey research, whether it be 
national, regional, or local, is conducted by telephone. This is the result of improved 
sampling and interviewing techniques, enhanced quality control, and the availability of a larger 
pool of information on what works or does not work when interviewing by telephone (Frey, 
1989). 

Telephone surveys provide fast, reliable data and afford an unprecedented opportunity to 
control the integrity of the sample as the study progresses. It is thus not surprising that 
telephone survey methodology has achieved a respected status as a means of gathering 
information to aid effective decision making in both the public and private sectors. Despite the 
higher cost of telephone surveys, the telephone survey approach is to be preferred over the 
mail survey approach inasmuch as it yields a higher response rate and is the only approach that 
allows those administering it to clarify its general intent and to elaborate on specific survey 
items as soon as questions arise (Michigan, 1981). 

Identifying the Population 

Whenever a telephone survey project is initiated, the first order of business should always 
be to determine exactly who should be surveyed and where to secure accurate names and 
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phone numbers for that group. This phase of the project is easily completed when the project 
involves students or former students for whom accurate information is presumed to be 
available in the college’s database. Identifying the population to be surveyed is clearly more 
of a challenge when the general public or employers are involved. In many instances, 
purchased lists are the most viable option and cost in the neighborhood of $300 to $500. 

These lists can be custom ordered as a random sample stratified by age, gender, ethnicity, ZIP 
code, size of firm, type of firm, or by a variety of other variables to meet the criteria for the 
project. 

Sampling 

Two common methods for sampling a population are the standard random sample and the 
random number dialing scheme. The standard random sample is generally produced either 
through a computer program designed to generate a random sample from an existing database, 
or by going through a listing such as a telephone book and selecting every “nth" individual, 
beginning from some randomly selected starting point. The random number dialing system is 
simply that-dialing a series of numbers at random utilizing a predetermined set of telephone 
prefixes within the service area being sampled. Computer programs are usually employed to 
generate the random number to be dialed. 

Several factors should be borne in mind when conducting a telephone survey utilizing a 
standard random sample. More and more individuals are opting for unlisted telephone 
numbers, excluding them from the pool of potential respondents. Others have multiple access 
lines coming into the same household, thus exhibiting a greater probability of being sampled 
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than the more typical household with only one access line. Still others have no telephone in 
the home, although there is some doubt that this group is either large enough or differs 
substantially enough from the general population to warrant special consideration. At least one 
study found that on many attitudinal questions those in households without telephones as a 
group did not differ to any great extent from the rest of the population as a whole (Lavrakas, 
1987). 

Caution also should be exercised when a random number dialing scheme is used for 
sampling because of the greater probability of multiple access households being sampled than 
the more typical household with only one access line. Surveys conducted by the Northwestern 
University Survey Laboratory in the Chicago metropolitan area in the mid-1980's consistently 
found 12% to 15% of households reporting more than one telephone number, which 
corresponded closely with Illinois Bell estimates (Lavrakos, 1987). Unquestionably the 
percent is much greater now, considering the growing popularity of separate telephone lines to 
accommodate modems for home computers. Another problem in efficient utilization of 
random number dialing is the large and growing number of households with FAX machines, as 
they increase costs due to the time and effort expended in eliminating them from the process. 
Designing the Telephone Survey 

Any good survey is a major research project, involving considerable time and resources. 

To make sure your efforts pay off, it is crucial that you spend some time planning (Suskie, 
1992). Designing any survey is a matter of style, but following standardized procedures will 
prevent an excessive compromise of data quality in the name of research artistry. Questions to 

7 

o 

ERIC 



6 



Telephone Surveys 



be addressed should include, “What are the objectives of your survey?” (what do you hope to 
gain as a result of this project?); "What do you really need to know (get rid of the "nice to 
know” stuff); "How will the results be used?” (in other words, what are you going to do with 
the data-if unsure, perhaps you need to revisit the "need to know” question); "How will you 
analyze and report the data?”; and “How much will the project cost?” 

The format for telephone surveys differs considerably from other types of surveys, and 
attention to the details of telephone survey design enable the interviewers to conduct each 
interview in as efficient and professional a manner as possible. Telephone interviews should 
always begin with an introductory statement by the caller explaining who they are and who 
they represent, the purpose of the study, the confidentiality of the data, and how long the 
interview should take. When appropriate, a disclaimer should also be read explaining that no 
one will try to sell the respondents anything as a result of their participation in the study. 

Any qualifying questions should follow the introduction, and naturally depend on the study 
guidelines. Some examples of items that are often used to qualify potential respondents are the 
year their program was completed at the college or how long the respondent has been a 
resident of the area. Clearly if the study is aimed at eliciting responses only from completers 
for a specific academic year, or only from residents who have been in the area for 5 years or 
more, for example, the interview with an individual not meeting those guidelines must be 
terminated. When doing so, it is appropriate to provide an explanation of why they are being 
eliminated as potential respondents and an expression of appreciation for their time. 
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The first questions on a telephone survey set the tone for the remainder of the interview. 
Thus, they should flow comfortably from the introductory remarks and build from simple to 
complex in nature. The first questions should establish the “rapport effect" (Frey 1989) where 
the respondent begins to feel a sense of trust in the interview situation and a willingness to 
cooperate to the fullest and most truthful extent. Dillman (1978) suggests that the first 
question be a closed item relevant to the topic, interesting, and easy to follow. The second 
question may then be open-ended, allowing the respondent to express a view and to find his or 
her “telephone voice." As with all survey design, questions of a sensitive nature (like those 
requesting demographic data) should be introduced last. 

Because the telephone survey is designed to be read, special instructions to interviewers 
should be easily recognizable. One effective means of accomplishing this is to type 
instructions in bolded capital letters, underlined and enclosed in parentheses or a box. This 
helps the interviewer to avoid the embarrassment of reading items not meant for the 
respondent. Another design feature proven to be most helpful to telephone interviewers is a 
column indicating the skip pattern for the questions. Inserting a broken line between entries 
calls immediate attention to the fact that a skip may be required (see Example 1). 

Naturally, any newly designed telephone survey should be field-tested to check the flow of 
the questions, the appropriateness of the instructions to interviewers, and the time required to 
complete an interview. This is easily accomplished by calling a few friends and administering 
the survey to them over the telephone. 
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Example 1 



OURS TI ON 


RESPONSE 


CODE 


SP 


1. Was JCCC your first choice when you 


Yes 


1 


3 


decided to attend college to pursue 
this major? 


No 


2 


2 


2. Which college was your first choice? 
(RECORD ACTUAL RESPONSE) 









3. How many semesters were you enrolled 
in that program at the community 
college? 

(RECORD ACTUAL RESPONSE! 

Print telephone surveys on both sides of standard 20# white paper in a manner that allows 
the questions on the front of each page to be read first. Upon reaching the end of the series of 
questions on the front of the pages, the interviewer then turns over the entire survey at once to 
continue the sequence of questions printed on the backs of the pages. This method negates the 
necessity of turning over page after page as the interview progresses, and has proven beneficial 
in assisting the caller in keeping his or her place as the interview progresses. It provides a 
considerable savings in the sheer weight and volume of the surveys compared to printing on 
heavier paper, one side only, and facilitates transportation of the blank surveys to the offices 
of the interview team and then bringing completed surveys back to the college. It also 
conserves space for the period of time after publication of the final report that completed 
surveys are retained for documentation purposes. 
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Methodology 

Telephone surveys can be used as a single method of collecting data, in conjunction with a 
mail survey to reach a larger group of potential respondents with less cost than the telephone 
survey alone, or as a follow up to mail surveys to substantially boost response rates. The 
nature of the project should determine which methodology is appropriate. 

An example of a project in which telephone methodology was used exclusively is the 
college’s community perception study which was conducted in 1987 and again in 1992. In 
each of these studies, a list was purchased of a random sample of 2,000 county residents over 
age 18, twice the number with whom completed interviews were anticipated. The list was 
stratified by ZIP code to insure inclusion of a reasonable number of respondents from outlying 
areas. As the study progressed, the integrity of the sample was maintained by substituting a 
new resident from the same ZIP code for each inaccessible individual. Inaccessibility resulted 
from a disconnected or incorrect telephone number, or simply from the inability to reach the 
individual at home despite calls initiated at different times of the day over a one-week period. 

A telephone survey in conjunction with a mail survey was used successfully for a Business 
and Industry Institute needs assessment conducted in 1992. In this instance, the purchased list 
consisted of all area firms with 50 or more employees, a random sample of one in three firms 
with 11-50 employees, and a random sample of one in ten firms with ten or fewer employees. 
All 426 firms with 50 or more employees and approximately half of the sample of 471 firms 
employing 11 to 50 workers were telephoned. The remaining firms with 11 to 50 employees 
and all 573 in the sample of firms with fewer than 1 1 employees were mailed surveys. 
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Of interest is the fact that although a total of 582 businesses responded to the study, only 
67 completed mail surveys. Clearly had telephone surveys not been utilized, the data would 
have been very limited. Despite the poor response rate for the mail surveys, it was not 
considered a wasted effort due to its marketing value. Specifically, the survey brought the 
numerous possibilities for classes and customized training available through the college to the 
attention of various employees of the firms receiving it. 

Conducting telephone interviews as a follow up of nonrespondents to mail surveys is the 
third means by which telephone surveys have enhanced the college’s research efforts. The 
college has been conducting follow-up studies of career program completers throughout its 28 
year history, and by far the best response rates have been in recent years since the initiation of 
the use of a telephone follow up of nonrespondents to the mail surveys. 

Mail/Telephone Comparisons 

Of particular interest to researchers are results of comparisons between mail survey 
respondents and nonrespondents who were later telephoned for an interview. These 
comparisons were derived from results of annual follow-up studies of the college’s career 
program completers and collected over time to enhance the reliability of the findings. Females 
and older completers were consistently more likely to return mail surveys than males and 
younger completers. What was somewhat of a surprise (and most gratifying) were the 
comparisons pertaining to completers’ success— the percent employed in related jobs and the 
average hourly wage of those working full-time in a related job. Year after year, essentially 
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no difference was found on either of these variables between initial mail respondents and 
nonrespondents who completed the telephone version of the survey (see Table 1). 



Insert Table 1 



Another question these comparisons answered was whether there were any substantial 
differences between initial mail respondents and nonrespondents who were telephoned in terms 
of the percent who indicated satisfaction with their full-time related job and with the college. 
Clearly, those who were telephoned were at least as satisfied as those who responded to the 
original mail survey. Due to space limitations, findings on satisfaction with the college are 
presented for the class of 1994-95 only— the most recent study for which data were available 
(see Table 2). 



Insert Table 2 

Note that when differences were evident in job success (based on the percent working in a 
related job and their average hourly wage) or in respondents’ satisfaction with their job and 
college experiences, generally a greater percent of nonrespondents who were telephoned than 
initial mail survey respondents were successful and satisfied. 

Telephone Survey Costs 

Cost is always a factor when determining appropriate methodology for any project, and 
telephone surveys are expensive. Clearly the project will be less expensive if the calling could 
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be done “in-house”, but unfortunately most colleges and universities have neither the staff nor 
the facilities for successfully completing this task. Even in situations where sufficient staff are 
available, problems may arise due to utilizing callers unskilled in the fine points of telephone 
survey work. Therefore, if telephone survey methodology is to be employed, in most cases an 
outside firm will need to be hired to do all or part of the work involved. 

There are three facets to telephone research— technical services, field services, and 
professional services. Most responses to telephone surveys are entered directly into the 
computer these days, and technical service personnel write the computer programs and extract 
the data. Field services people conduct the actual telephone interviews, and professional 
services personnel write the reports and may also design the survey and make a presentation of 
findings, complete with charts and various other visual aids. 

As a general rule of thumb, in the Midwest for just technical and field services you can 
expect to pay approximately $1 per minute for each completed survey, or $8 to $10 for a 
typical 8 to 10 minute interview, including some long distance. These costs will be higher in 
Florida, Texas, or on either coast (closer to $1.25 per minute), and may be somewhat lower in 
the South. If no long distance is involved, the cost drops by approximately 10 cents a minute. 
The number of completed surveys required is a strong determinant of the total cost since the 
“up front" work is most often the same. Expect the cost per completed survey to decrease as 
the number of potential responses increases. 

The cost of technical services can be eliminated if you choose to have the data from 
telephone interviews entered on a paper survey, but this necessitates having someone do data 
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entry for you. If you contract for the technical services but wish to economize by not utilizing 
an outside firm’s professional services, you will receive your data on a disk and then must 
design your own survey and either do the analysis and report writing yourself or assign these 
tasks to someone else at your institution. 

Computer-driven telephone interviews are a wonderful use of technology, but tend to be 
inappropriate and unduly expensive if the survey you design requires a substantial number of 
verbatim comments or involves information the interviewer will need to look up to determine a 
code, such as, “Which college was your first choice?”. Writing out comments or looking up 
codes after terminating the call inflates the average cost by approximately $1.00-$ 1.25 per 
survey for each minute the interviewer must spend accomplishing these tasks. 

In the Midwest, costs for professional services can run anywhere from about $600 for a 
“top line” report (very basic results or printouts) to $10,000 or more for a “full" report which 
may include designing the survey and providing a presentation of results, complete with visual 
aids and nicely bound copies of results. In between these two possibilities is the “executive” 
report. Again, costs will be higher in Florida, Texas, or on either coast and slightly lower in 
the South. 

The college has chosen to continue utilizing the paper telephone survey after one rather 
disastrous experience with computerized survey administration. Thus, the survey design, data 
entry, analysis, and report writing for all of the college’s telephone survey work is completed 
in-house. The low bid received by the college in summer of 1996 for field work only, based 

15 

o 

ERIC 



14 



Telephone Surveys 



on completing approximately 700 interviews of 8 to 12 minutes each with JCCC short- and 
long-term career program completers was $8.21 per survey. 

Training the (Professional) Interviewers 

It is imperative that all interviewers be adequately trained in telephone survey techniques 
during an orientation session prior to initiation of the study. Regardless of the experience of 
the interview team, basic instructions should be reviewed, such as the necessity of reading all 
questions exactly as stated with no paraphrasing, remaining neutral and avoiding any positive 
or negative feedback, writing down all verbatim comments exactly as stated, sitting up straight 
and smiling so the voice reflects interest and regard for the respondent, and completing the call 
record appropriately. 

During the orientation, the entire survey should be reviewed in detail to insure that every 
interviewer is intimately familiar with the qualifying questions, the instructions to callers 
which are not to be read aloud, the skip patterns, and the manner in which every question is to 
be read. The most efficient way to do this is to have the interviewers take turns participating 
in a mock interview as you go through every item on the telephone survey. This method 
familiarizes the interviewers with the survey instrument, gives them an opportunity to ask 
questions about the study, and helps them with any problems they may encounter with 
pronunciation of some of the words and phrases contained in the survey. 

It is highly recommended that spot checks be conducted throughout the interview process, 
and especially at the very beginning. Completed surveys should be picked up a day or two 
after the telephoning has begun and calls made to a few randomly selected respondents to be 
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sure the interviews had been conducted in an efficient, professional manner. This step is 
particularly important when using the services of a market research firm with which you are 
unfamiliar. Be especially cautious about hiring a firm that contracts out the calling to 
individuals working out of their homes. It is immensely tempting for these interviewers to 
complete surveys without making the actual calls in order to maximize their production. Your 
spot check shortly after the project begins and throughout the course of the study should 
uncover any such problems 
Conclusions 

Telephone surveys are effective tools to substantially enhance response rates while 
providing fast, reliable data and the opportunity to control the integrity of the sample as the 
study progresses. However, they are expensive. Basically, three things will govern the 
ultimate success of any telephone research project: the accuracy of the list of potential 
respondents, the design of the telephone survey, and the expertise of the telephone 
interviewers. In other words, quality data is the result of asking the right people the right 
questions in the right way. It’s as simple as that. 
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Table 1 

Initial Mail vs. Follow- up Telephone Response; 
Questions Pertaining to Job Success 



Item 


Mail Responses 


Telephone Responses 


Percent Employed in Related Job 


Class of 1994-95 


78.9% 


83.2% 


Class of 1993-94 


78.1 


83.9 


Class of 1992-93 


85.2 


80.8 


Class of 1991-92 


75.2 


83.1 


Class of 1990-91 


78.2 


73.0 


Average Hourly Wage 1 


Class of 1994-95 


$12.63 


$12.45 


Class of 1993-94 


12.55 


11.95 


Class of 1992-93 


11.87 


11.86 


Class of 1991-92 


11.42 


11.50 


Class of 1990-91 


11.07 


11.06 



Note. 1 Results detail responses for respondents employed in a full-time related job only. 
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Table 2 

Initial Mall vs. Follow-up Telephone Responses : 
Questions P ertaining to Satisfaction 



Item 


Mail Responses 


Telephone Responses 


Satisfaction With Job 1, 2 


Class of 1994-95 


59.0% 


76.5% 


Class of 1993-94 


83.3 


83.1 


Class of 1992-93 


79.5 


91.1 


Class of 1991-92 


64.3 


77.1 


Class of 1990-91 


75.3 


93.6 


Satisfaction with JCCC: Class of 1994-95 


JCCC First Choice to Attend 


90.3% 


92.6% 


Would Attend JCCC Again 


91.2 


95.4 


Would Recommend JCCC to Friends 


96.0 


96.5 


Would Encourage Own Children to 


Attend JCCC 


80.6 


89.8 



Note . 1 Results detail responses for those employed in a full-time related job only. 

2 Data were collected utilizing a 5-point scale ranging from very satisfied to very 
dissatisfied. Thus, the “satisfied” data include “very satisfied” and “somewhat 
satisfied” responses combined. 
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